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FOREWORD 


This  report  documents  a  Review  &  Analysis  of  Distributed,  Collaborative 
Decision-Making ,  as  it  affects  future  Air  Force  personnel  and  training  issues. 
The  task  was  performed  by  the  Crew  System  Ergonomics  Information 
Analysis  Center  (CSERIAC)  for  Armstrong  Laboratory,  Cognition  and 
Performance  Division,  Force  Planning  and  Management  Technology  Branch 
(AL/HRCF).  It  was  conducted  under  Department  of  Defense  (DoD)  Contract 
Number  SP0900-94-D-0001,  and  the  CSERIAC  subscription  account 
number  891 AZ2701  93704-27.  The  CSERIAC  Director  during  this  period 
was  Mr.  Donald  A.  Dreesbach.  The  primary  human  factors  analyst  and 
author  was  Barbara  Palmer,  who  was  assisted  by  human  factors  analysts 
Joyce  A.  Cameron  and  Jennifer  I.  Soest.  Mark  A.  Cummings  assisted  with 
document  preparation  and  distribution.  These  analysts  worked  under  the 
direction  of  the  CSERIAC  Project  Manager  and  Senior  Technical  Analyst, 
Frank  C.  Gentner.  Joyce  Cameron  and  Frank  Gentner  developed  the  briefing 
in  Attachment  A,  based  on  the  report  and  other  related  materials. 


EXECUTIVE  SUMMARY 


The  future  battlefield  will  be  characterized  by  complex  and  dense  information  flows,  with 
operators  and  decision-makers  at  times  in  close  proximity,  and  in  other  situations,  distant  from  one 
another.  Decisions  will  be  made  by  groups  of  people  often  geographically  separated  from  one 
another.  This  Review  &  Analysis  describes  how  such  decisions  are  made,  and  presents  research 
regarding  how  good  group  decision-makers  can  be  selected  and  trained. 

As  the  Air  Force  prepares  for  the  decades  ahead,  decisions  need  to  be  made  now  to  select  and 
prepare  tomorrow’s  decision-makers.  This  document  reviews  planning  documents  of  the  Air  Force 
and  the  Department  of  Defense,  and  scientific  and  technical  reports  regarding  the  scientific  study  of 
decision-making,  to  paint  a  picture  of  the  types  of  tasks  future  Air  Force  decision-makers  will 
perform,  and  how  they  will  make  decisions. 

This  Review  &  Analysis  examined  how  the  Air  Force  could  exploit  the  scientific  findings 
regarding  decision-making  as  it  lays  plans  for  the  future  battlespace  and  other  military  operations. 
The  most  important  messages  of  this  Review  &  Analysis  for  the  Air  Force  are  the  following: 

•  Research  on  the  differences  between  group  and  individual  decision-making  indicates  that  the 
biggest  benefit  of  collaborative  group  activity  is  the  group’s  increased  cognitive  resources, 
compared  to  that  of  a  single  individual. 

•  The  greatest  detriment  to  collaborative  distributed  decision-making  is  that  we  must  rely  on 
technology  rather  than  face-to-face  interactions,  and  subtleties  of  human  communication  may  be 
lost. 

•  The  best  predictor  of  good  decision-making  is  the  experience  of  the  decision-maker,  the 
implication  being  that  training  for  decision-making  is  paramount. 

•  While  future  indices  of  personality  and  other  selection  variables  may  have  better  success,  current 
tools  do  not  offer  a  great  deal  of  information  about  who,  in  the  general  sense,  will  be  a  good 
decision-maker. 

•  All  the  military'  services  are  engaged  in  research  into  what  variables  are  important  in  a  group 
decision-making  training  program. 

•  A  group  decision-making  training  program  is  outlined,  which  focuses  on  communication  skills, 
situation  awareness,  and  decision-making  skills. 


1.0  INTRODUCTION 


This  Review  &  Analysis  focuses  on  concepts  involved  in  collaborative  distributed  decision-making. 
Also  covered  will  be  factors  that  research  has  shown  to  contribute  to  the  success  of  a  group’s 
decision-making  processes.  Before  we  address  some  of  these  decision-making  topics,  a  discussion  of 
the  elements  of  the  future  battlespace  and  some  of  its  communication  pathways  may  help  establish  a 
context  within  which  this  research  will  be  relevant. 

1.1  DECISION-MAKING  IN  THE  CONTEXT  OF  THE  FUTURE  BATTLEFIELD 

As  we  look  at  various  views  of  the  future  battlefield,  it  will  be  clear  why  the  Air  Force  is  concerned 
about  effective  distributed  collaborative  decision-making.  The  future  battlespace  is  painted  for  us  in 
detail  by  joint  service  and  Air  Force  vision  documents  such  as  Air  Force  2025  (Air  University,  Air 
Education  and  Training  Command,  1996),  Joint  Vision  2010  (Chairman  of  the  Joint  Chiefs  of  Staff, 

1 997),  and  the  Advanced  Battlespace  Information  System  (ABIS)  Task  Force  Report  (Jones  & 
Cebrowski,  1996).  Main  themes  are  that  joint  forces  will  work  in  even  closer  coordination  and  that 
decisions  will  be  made  in  geographic  locations  distant  from  battle.  The  emphasis  on  groupwork  and 
the  geographical  distance  between  decision-makers  and  the  scene  of  action  provides  the  motivation 
for  this  Review  &  Analysis  on  distributed  collaborative  decision-making.  The  critical  nature  of  the 
decisions  made  by  these  groups  is  the  key  to  battlespace  dominance.  No  matter  which  view  of  the 
future  battlespace  turns  out  to  be  most  accurate,  all  views  of  the  future  battlespace  make  it  clear  that 
this  will  be  an  information-age  military.  Superior  weapons  systems  will  no  longer  be  sufficient. 

Those  who  can  control  the  flow  of  information  will  have  a  certain  advantage. 

As  with  all  complex  decisions,  decisions  in  the  future  battlefield  will  be  heavily  information- based. 
Information  gathered  from  satellites,  AWACS,  and  UAVs  will  flow  to  ground  stations,  tanks,  and 
aircraft.  Battles  will  be  waged  in  a  manner  unlike  any  other  time,  and  will  be  characterized  by 
distributed  decision-making  and  geographically  disparate  personnel.  The  authors  of  Air  Force  2025 
(Air  University,  Air  Education  and  Training  Command,  1996)  indicate  that  future  battles  may  not  be 
about  the  capture  of  land;  rather  they  may  be  more  space-  and  cyberspace-focused.  Satellites  and 
space-based  sensors  will  be  important,  as  will  multi-function  uninhabited  air  vehicles  and  space 
planes  (transatmospheric  vehicles).  Joint  operations  will  involve  air,  land,  and  naval  cooperation. 
Information  about  the  location  of  friendly  forces,  enemy  forces,  weather,  logistics,  and  terrain  data 
must  be  gathered,  stored,  channeled  through  a  common  operating  environment,  made  available  to 
decision-makers  as  needed,  and  accompanied  by  intelligent  agents  whose  work  it  will  be  to  minimize 
the  information-processing  burden  of  the  user.  The  discipline  of  human  factors  will  also  be  necessary 
in  the  creation  of  this  interface  between  the  information  space  and  the  user.  This  huge  volume  of 
information  will  be  presented  in  a  useable  way  to  the  decision-maker  via  information  fusion, 
groupware,  and  virtual  reality  techniques.  The  mass  of  information  gathered  in  the  future  battlespace 
will  be  enormous,  and  intelligent  agents  (software  such  as  decision  aids)  will  be  needed  to  help  refine 
the  users’  queries  so  that  the  right  information  can  be  delivered  at  the  right  time  without 
overburdening  the  user. 

Before  a  discussion  of  the  discipline  of  decision-making,  an  example  of  a  future  war  room  will  help 
put  the  topic  of  collaborative  distributed  decision-making  in  a  relevant  context  for  Air  Force 
personnel  strategists.  By  the  year  2025,  a  virtual  reality,  holographic  representation  of  a  battle  scene 
will  be  available  to  the  Commander-in-Chief  and  others  engaged  in  battleground  decision-making. 
Weapons  sensors  and  surveillance  equipment  will  feed  information  in  near  real-time  through  the 
information  grid,  so  that  it  will  be  available  on  an  as-needed  basis  to  military  planners.  In  this  vision, 
the  distributed  and  collaborative  nature  of  future  military  decision-making  is  clear. 
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Air  Force  2025  (Air  University,  Air  Education  and  Training  Command,  1996)  describes  the  future 
Commander  in  Chief  (CINC)’s  battle  management  operations  center  as  an  air  operations  center.  The 
Commander  in  Chief  and  the  Joint  Force  Air  Component  Commander  (JFACC)  will  have  a  God’s 
eye  view  of  the  battlespace  made  available  through  a  highly  sophisticated  information  processing 
network.  The  entire  battlespace  can  be  viewed,  or  the  commander  can  zoom  to  a  specific  location,  in 
a  manner  similar  to  that  viewed  in  Figure  1,  a  holographic  war  room.  This  technology  will  allow 
monitoring  and  engagement  through  the  area  of  responsibility,  down  to  a  specific  target,  allowing  the 
JFACC  to  interact  with  the  holographic  depiction,  controlling  the  engagement  and  monitoring  the 
battle  progress  in  near  real  time. 


Figure  1.  Holographic  warroom  of  the  future. 

Source:  http://www, afit.af.mil/Schools/PA/gall3.htm,  courtesy  of  Gene  Lehman,  Air  Force  Institute 
of  Technology  (AFIT/LSEC). 

The  battle  command  center  will  allow  real-time  centralized  control.  Veiy  high-speed  computers  will 
show  battle  simulations,  forecast  outcomes  in  accelerated  time  modes,  and  assess  battle  damage. 
Laser  and  microwave  weapons  will  operate  at  the  speed  of  light.  Different  courses  of  action  will  be 
evaluated  via  accelerated  simulations.  The  capabilities  and  doctrines  of  enemy  forces  will  be 
simulated  through  knowledge-based  artificial  intelligence  software  and  the  JFACC  can  mix  weapons 
and  weapons  effects.  A  series  of  options  can  be  simulated  in  this  war  room,  and  a  course  of  action 
chosen.  The  commander  then  can  send  orders  to  the  individual  weapons  systems  for  execution.  All 
the  systems  will  provide  feedback,  ensuring  rapid  retargeting  so  that  the  CINC’s  and  the  JF ACC’s 
objectives  are  met. 
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1.2.  PURPOSE 


Consider  the  downing  of  an  Iranian  airliner  by  the  USS  Vincennes,  the  Three  Mile  Island  nuclear 
reactor  incident,  and  the  recent  crash  of  an  airliner  in  Colombia,  and  the  catastrophic  consequences  of 
human  error  are  clear.  The  complexities  of  modem  technology  and  political  situations  place 
unprecedented  demands  on  the  human  operator.  As  technological  sophistication  increases,  decision¬ 
making  will  become  even  more  complex  (Salas  &  Cannon-Bowers,  1996).  Several  trends  in  today’s 
world  dictate  the  need  for  scientists  to  make  advances  in  understanding,  developing  training,  and 
supporting  effective  decision-making.  Trends  such  as  advancing  technology,  the  complexity  of 
decision  environments,  and  the  increasing  cost  of  decision  errors  make  it  evident  that  this  research  is 
needed  (Cannon-Bowers,  Salas,  &  Pruitt,  1996).  As  Norman  (1993)  pointed  out,  people  are  facing 
more  cognitively  demanding  tasks  at  work  and  play,  and  the  consequences  of  poor  or  ineffective 
performance  are  becoming  more  costly,  in  terms  of  money  and  loss  of  life.  Aviation  accidents, 
military  mishaps,  and  industrial  incidents  are  often  caused  by  human  error  (Cook  &  Woods,  in  press). 
Fortunately,  there  has  been  a  resurgence  of  activity  during  the  last  ten  years  on  the  topic  of  decision¬ 
making,  especially  in  the  context  of  real,  complex,  time-compressed,  stressful  settings.  This  paper 
will  focus  for  the  most  part  on  the  decision-maker  rather  than  on  the  process  of  decision-making,  but 
we  must  first  examine  the  topics  being  studied  under  the  aegis  of  decision-making  so  that  we  are 
aware  of  the  tasks  and  constraints  facing  decision-makers.  Some  very  brief  mention  of  the  theoretical 
thrusts  will  lead  to  a  discussion  of  the  state  of  research  into  how  to  select  good  decision-makers  and 
how  to  train  for  effective  decision-making. 

The  purpose  of  this  Review  &  Analysis  is  to  introduce  topics  in  the  realm  of  decision-making  and  to 
point  out  their  relevance  to  communications  that  are  likely  in  the  future  battlespace.  Examples  of 
current  group  decision-making  training  programs  are  presented,  and  recommendations  for  future 
research  and  development  are  made.  Implications  for  future  AF  personnel  and  training  issues  are 
indicated. 

1.3  DEFINITIONS 

The  topic  of  this  Review  &  Analysis  is  Distributed  Collaborative  Decision-Making.  A  definition  of 
some  important  terms  is  in  order;  the  terms  involved,  decision-making,  distributed,  and  collaborative, 
will  be  defined.  Later  in  Section  3,  these  topics  and  their  relevance  to  future  military  decision¬ 
making  are  developed  in  greater  detail. 

1.3.1  Decision-Making 

What  is  involved  when  an  individual  or  group  makes  a  decision?  Decision-making  as  a  term  covers  a 
lot  of  territory.  In  general  terms,  it  can  be  said  that  decision-makers  (1)  define  the  problem,  (2) 
identify  and  analyze  possible  courses  of  action,  and  (3)  select  a  course  of  action.  In  Section  3, 
strategies  that  people  use  to  make  decisions  are  reviewed  briefly.  Other  topics  to  be  addressed  are 
how  group  decision-making  differs  from  individual  decision-making  and  how  distributed  decision¬ 
making  differs  from  collocated  decision-making.  It  should  be  noted  here  that  decision-making  is  a 
term  that  can  cover  a  wide  range  of  cognitive  behaviors,  and  this  most  broad  usage  is  probably 
appropriate  for  this  Review  &  Analysis.  That  is,  people  working  in  teams  in  the  future  battlefield  will 
be  engaging  in  planning,  generation  of  possible  courses  of  action,  and  analyses  of  battle  damage,  all 
of  which  contribute  to  decision-making.  So  this  Review  &  Analysis  will  discuss  “decision-making”  in 
the  widest  of  all  possible  senses.  Just  as  solution  generation  follows  a  definition  of  the  problem, 
awareness  of  the  problem  precedes  definition  of  the  problem,  so  situation  assessment  and  situation 
awareness  will  be  addressed  in  this  Review  &  Analysis  as  well. 
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1.3.2  Collaborative  Decision-Making 

Collaborative  decision-making  implies  the  existence  of  more  than  one  decision-maker  who  work 
together  whether  a  group  or  team.  Orasanu  and  Salas  (1993)  state  that  a  team  consists  of  two  or  more 
individuals  who  rely  on  more  than  one  source  of  information,  share  a  set  of  common  goals,  and  have 
defined  roles  and  responsibilities.  There  is  interdependence  and  coordination  among  members. 

Teams  are  also  characterized  by  adaptive  management  of  the  team’s  resources.  Team  members 
possess  task-relevant  knowledge.  Groups  are  distinguished  from  teams  in  that  teams  are  more 
differentiated  and  interdependent,  whereas  groups  are  more  homogeneous  and  the  members  more 
interchangeable.  Since  the  future  battlefield  will  contain  a  variety  of  decision  environments,  the 
distinction  between  groups  and  teams  will  not  be  emphasized  in  this  Review  &  Analysis.  These  terms 
will  be  used  interchangeably. 

1.3.3  Distributed  Decision-Making 

Several  aspects  of  a  team  can  be  distributed.  Authority,  responsibility,  task  knowledge,  and  expertise 
are  usually  inherently  distributed  in  all  teams.  For  instance,  one  person  is  usually  in  charge,  so 
authority  is  unequal.  Responsibilities  are  assigned,  and  task  knowledge  and  expertise  are 
differentiated.  This  leaves  location  as  the  defining  characteristic  of  distributed  teams.  A  distributed 
team,  in  the  context  of  this  Review  &  Analysis  and  most  of  the  scientific  literature  (Orasanu  and  Salas, 
1993),  is  a  team  with  one  or  more  members  geographically  separated  from  the  others. 
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2.0  APPROACH 


As  military  decision-makers  prepare  to  work,  the  task  they  face  has  many  components.  Decision¬ 
makers  will  be  required  to  work  together  as  a  team,  and  may  be  working  in  war  rooms  and  ground 
stations  far  apart  from  one  another.  They  will  be  facing  extreme  levels  of  stress  from  the  critical 
nature  of  their  mission,  and  the  time  factor  will  be  extraordinary.  Given  these  conditions,  how  can 
good  decisions  be  made?  Are  some  people  better  decision-makers  than  others?  Can  the  Air  Force 
use  information  about  who  makes  a  good  decision-maker  in  its  selection  criteria?  Can  effective  group 
decision-making  strategies  be  taught?  Are  good  decision-makers  bom  or  made?  What  are  the  steps 
that  groups  undertake  as  they  reach  consensus?  What  is  the  state  of  the  science  regarding  group 
decision-making?  This  Review  &  Analysis  sought  to  answer  these  questions  by  analyzing  an 
overview  of  the  literature  on  collaborative  distributed  decision-making  and  the  future  battlefield. 

Information  for  this  Review  &  Analysis  was  gathered  from  several  sources.  The  bulk  of  information 
was  derived  from  Air  Force  and  published  scientific  literature,  including  books,  technical  reports,  and 
journals  from  the  fields  of  military  forecasting,  decision-making,  team  processes,  and  communication. 
Relevant  literature  was  identified  subsequent  to  a  comprehensive  computerized  search  of  the 
literature.  Literature  searches  were  performed  on  several  databases,  including: 

•  Defense  Technical  Information  Center  (DTIC)  Technical  Reports  (TR) 

•  DTIC  CD-ROM 

•  DTIC  Work  Unit  Information  Summary  (WUIS) 

•  National  Aeronautics  and  Space  Administration  Remote  Control  (NASA  Recon) 

•  PsycINFO© 

•  National  Technical  Information  Service  (NTIS) 

•  Dissertation  Abstracts  Online 

•  Books-in-Print 

•  Educational  Resource  Information  Center  (ERIC) 

•  Jane’s  Defense  and  Aerospace 

Relevant  and  recent  literature  and  researchers  were  identified  and  this  information  was  used  to  access 
other  sources.  From  over  a  thousand  citations  (see  Volumes  2  and  3),  journal  articles,  technical 
reports,  and  book  chapters  from  the  past  decade,  created  in  or  translated  into  English,  were  obtained, 
reviewed,  and  analyzed  for  this  report.  In  addition  to  databases  of  literature,  additional  information 
was  obtained  through  World  Wide  Web  newsgroups,  internal  Air  Force  instructions  and  other 
documents,  subject  matter  experts,  and  electronic  documents. 
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3.0  FINDINGS 


3.1  THE  STUDY  OF  DECISION-MAKING 

This  section  will  describe  the  steps  involved  in  decision-making,  discuss  some  of  the  major 
theoretical  frameworks  regarding  decision-making,  and  present  the  advantages  and  disadvantages  of 
collaborative  and  distributed  decision-making.  Then,  literature  about  what  makes  an  effective 
decision-making  group  is  presented.  This  literature  will  be  examined  according  to  three  thrusts — how 
the  Air  Force  can  control  three  kinds  of  variables.  Environmental  variables  (for  instance,  the 
availability  of  technology)  can  be  directly  controlled  by  the  Air  Force.  Factors  inherent  to  the 
individual,  traits,  and  aptitudes,  can  be  the  basis  for  Air  Force  selection  procedures.  Finally, 
behaviors  that  can  be  learned  can  be  used  to  design  decision-making  training  courses. 

3.1.1  Steps  in  Decision-Making 

Literature  abounds  which  lists  the  different  steps  involved  in  the  making  of  a  decision,  but  generally, 
decision-making  is  considered  to  involve  these  processes: 

•  Problem  Recognition 

•  Information  Gathering 

•  Development  of  Options 

•  Implementation  of  Options 

It  is  important  to  note  a  term  at  this  point  that  is  used  in  many  contexts,  and  to  define  what  it  means 
here.  The  Problem  Recognition  stage  can  be  said  to  incorporate  the  state  of  being  aware  of  a 
situation.  Situational  (or  Situation)  Awareness  (SA),  “the  capability  to  appropriately  assess  yourself, 
your  system,  and  your  environment  to  make  the  right  decision  at  the  right  time,”  is  a  critical  element 
of  decision-making  (Vidulich,  1992,  cited  in  Kokorian,  1995).  Endsley  and  Jones  (1997)  divide  SA 
into  these  three  high-level  stages: 

•  Perception  of  elements  in  the  environment, 

•  Comprehension  of  the  current  situation,  and 

•  Projection  of  future  status. 

The  resulting  information  from  these  steps  serves  as  a  major  input  to  decision-making.  In  fact,  SA  is 
often  considered  the  critical  component  of  decision-making  (Endsley  and  Jones,  1997;  Noble,  1993). 
Thus,  effective  decision-making  is  linked  to  effective  and  accurate  SA. 

This  discussion  of  the  stages  in  decision-making  give  us  some  familiarity  with  the  behaviors  of 
individuals  and  groups  involved  in  group  decision-making.  We  now  need  to  consider  the  differences 
between  individual  and  group  decision-making,  and  collocated  and  distributed  decision-making. 

3.1.2  Theoretical  Perspective 

A  brief  look  at  the  history  of  decision-making  shows  a  recent  leap  from  a  classical  to  a  naturalistic 
perspective. 
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3.1.2.1  Classical  Vs.  Naturalistic  Decision-Making 

The  classical  theories  were  characterized  by  normative  models  of  decision  behavior,  focusing  on  the 
correct  way  to  make  a  decision.  Normative  models  of  decision-making  focused  on  the  process  of 
option  generation  and  simultaneous  evaluation  to  choose  a  course  of  action  (Wright,  1984). 
Researchers  in  the  mid-1980s  who  were  facing  more  applied  issues  such  as  those  found  in  decision 
support,  system  design,  and  training,  were  thwarted  by  the  classical  approach,  which  was  seen  to  be 
narrow,  with  contrived  decision-making  situations  which  were  of  little  consequence  to  real  world 
decision-makers  (Cannon-Bowers  et  al.,  1996).  A  small  group  of  decision  researchers  met  in  Dayton, 
Ohio  in  1989  to  discuss  a  new  alternative,  naturalistic  decision-making  (NDM).  Naturalistic 
decision-makers  focus  on  the  description  of  how  people  make  decisions,  rather  than  a  prescriptive 
approach.  NDM  emphasizes  complex,  real-life  decision-making  environments.  An  edited  volume 
followed  this  conference  (Klein,  Orasanu,  Calderwood,  &  Zsambok,  1993).  NDM  has  made 
important  contributions  to  the  study  of  decision-making.  It  has  focused  attention  on  the  experience 
that  decision-makers  bring  to  bear  in  making  good  decisions.  NDM-related  models  indicate  that 
different  cognitive  strategies  are  used  for  static  versus  temporally  evolving  situations.  Also,  these 
models  indicate  that  action  and  perception  are  crucial  aspects  of  NDM.  They  state  that  human 
resources  are  limited,  and  that  human  decision-making  competence,  rather  than  dysfunction,  should 
be  emphasized  (Cannon-Bowers  et  al.,  1996). 

3.1.2.2  Characteristics  of  Naturalistic  Decision-Making  (NDM) 

NDM  occurs  in  situations  characterized  by  these  eight  factors  (Orasanu  &  Connolly,  1993,  p.  7), 
which  can  be  seen  in  many  current  and  future  military  scenarios: 

1 .  Ill-structured  problems 

2.  Uncertain  dynamic  environments 

3.  Shifting,  ill-defined,  or  competing  goals 

4.  Action/feedback  loops 

5.  Time  stress 

6.  High  stakes 

7.  Multiple  players 

8.  Organizational  norms  and  goals 

3.1. 2.3  Recognition-Primed  Decision-Making  Theory 

The  dominant  NDM  theory  is  Recognition-Primed  Decision-Making  (RPD)(Klein,  1989).  Klein  and 
others  base  this  theory  on  ten  years  of  observation  of  military  operations  and  other  operational 
command  situations.  RPD  asserts  that  those  in  command  generate  courses  of  action  immediately 
after  a  rapid  assessment  of  the  situation.  The  commander  may  often  recall  a  single  course  of  action 
once  employed  in  a  similar  situation.  Recognizing  the  similarity  of  one  situation  to  a  former  one  is 
the  key  to  RPD.  The  commander  may  then  run  through  the  chosen  option  by  a  mental  simulation  in 
order  to  assess  its  implications  before  putting  it  into  action.  Klein  (1989)  determined  that  80  percent 
of  fire  fighter  commanders’  decisions  were  recognition-primed.  To  the  decision-maker,  use  of  RPD 
feels  like  an  intuitive  response  rather  than  an  analytical  judgment  of  alternative  options.  The 
following  list  characterizes  tenets  of  RPD  (Klein,  1993): 

•  In  operational  settings,  people  try  to  find  the  first  course  of  action  that  works,  not  the  best  one. 

•  Decision-making  consists  of  two  aspects — assessing  the  situation,  and  selecting  a  course  of 
action. 

•  Experienced  decision-makers  can  usually  assess  the  situation  quickly  and  accurately. 
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•  Once  the  situation  is  understood,  the  course  of  action  is  usually  obvious. 

•  Decision-makers  often  must  be  prepared  to  act  without  fully  examining  the  parameters  and 
contingencies. 

•  Decision-making  and  problem-solving  are  inter-related. 

•  Decision-makers  arrive  at  a  course  of  action  by  generating  pertinent  opinions  rather  than  filtering 
out  unacceptable  ones. 

RPD  is  an  apt  model  for  military  decision-making  in  many  ways.  The  eight  characteristics  listed 
above  can  be  found  in  many  military  decision-making  environments.  Researchers  in  many 
operational/  incident  commanders  across  diverse  domains  have  found  Klein’s  work  to  be  applicable. 
In  a  study  of  officers  in  the  London  Fire  Brigade,  Hendry  and  Burke  (1995)  concluded  that  81  percent 
of  the  officers’  decisions  were  reached  without  appraisal  of  an  alternative  option.  Serfaty,  Macmillan, 
Entin,  and  Entin  (in  press)  used  a  naturalistic  approach  in  an  examination  of  course-of-action 
decisions  among  US  Army  commanders.  Pascual,  Henderson,  Femall,  Ahmed,  and  McGahan  (1994) 
found  that  naturalistic  strategies  accounted  for  87  percent  of  the  decisions  made  by  officers  of  a 
British  Army  command  post  simulator.  The  Tactical  Decision-Making  Under  Stress  (TADMUS) 
project,  examining  US  Naval  Commanders’  warfare  decision-making  processes,  with  the  objective  of 
designing  better  training  and  support  software,  has  also  adopted  an  RPD  framework  (Cannon-Bowers, 
Salas,  &  Grossman,  1991). 

3.1.3  Concepts  of  Collaborative  and  Distributed  Decision-Making 

These  stages  in  decision-making  give  us  some  familiarity  with  the  behaviors  of  individuals  and 
groups  involved  in  such  a  task.  We  now  need  to  consider  the  differences  between  individual  and 
group,  and  collocated  and  distributed,  decision-making. 

3.1.3.1  Collaborative  —Group  vs.  Individual 

This  section  reviews  the  differences  between  decision-making  by  an  individual  and  decision-making 
in  a  group  context.  As  we  move  from  what  is  known  about  individuals  and  decision-making,  Klein 
(1993)  says: 

The  idea  is  to  take  what  we  know  about  the  way  individuals  think,  and  bump 
it  up  one  level  as  a  model  of  teams.  A  cognitive  model  of  team  decision¬ 
making  views  a  team  as  an  intelligent  entity,  subject  to  all  the  cognitive 
limitations  of  an  individual — limited  memory,  limited  attention,  limited 
situation  assessment  capabilities,  and  so  on.  The  intent  of  the  cognitive 
framework  is  to  focus  attention  on  the  team,  rather  than  on  the  team 
members,  and  to  take  advantage  of  our  knowledge  of  individuals  to  better 
understand  team  decision-making. 

(p.  126) 

Another  view  of  group  versus  individual  decision-making  comes  from  a  study  of  aircrew  decision¬ 
making  (Korkorian,  1995).  Compared  to  decision-making  by  individuals,  cockpit  crew  decision¬ 
making  is  managed,  that  is,  the  captain  has  responsibility  for  making  decisions  but  is  supported  by 
other  crew  members,  and  outside  the  cockpit  by  air  traffic  controllers  and  dispatchers.  Team 
decision-making  is  better  than  individual  decision-making  because  there  is  greater  cognitive  capacity 
to  consider  a  larger  picture,  more  alternatives,  and  the  team  is  more  likely  to  identify  more  possible 
errors.  Team  members  may  feel  less  stress  when  the  stress  comes  from  public  scrutiny  of 
performance,  because  failures  of  team  members  may  be  masked  (Klein,  1993).  On  the  downside, 
group  decision-making  suffers  if  poor  communication  among  members  leads  to  erroneous  sharing  of 
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mental  models.  In  addition,  errors  can  breed  through  the  crew,  leading  to  false  collective  confidence. 
Klein  (1993)  indicates  that  ambiguity  can  cascade  through  teams,  since  no  member  can  be  sure  of 
understanding  how  others  are  interpreting  events.  Klein  (1993)  states  that  stressors  may  have  a 
greater  effect  on  teams  than  individuals,  since  the  stress  would  disrupt  team  interactions  as  well  as  the 
performance  of  individuals.  While  individuals  making  decisions  often  do  well  under  stress,  with 
many  studies  showing  little  effect  of  noise  and  time,  groups  may  be  susceptible  to  time  pressure. 

Time  pressure  may  lead  the  group  to  throw  off  coordination  among  members.  In  addition,  Klein  also 
states  that  individuals  can  use  recognition  decision  strategies  to  save  time,  but  groups  do  not  have 
such  shortcuts  available  to  them.  Clearly,  there  is  a  terrific  benefit  in  general  to  group  versus 
individual  decision-making,  since  the  collective  mental  abilities  and  task  knowledge  of  a  group  are  far 
superior  to  that  of  any  individual.  Auditory  noise  can  degrade  team  communication,  and  high 
workload  imposes  a  differentially  greater  burden  on  teams  than  individuals  because  coordination 
efforts  must  be  increased. 

3.13.1.1  Group  Planning — An  Example 

In  reality,  what  does  a  group  activity  look  like?  Who  might  participate?  Who  coordinates  with 
whom?  What  kinds  of  information  might  be  transmitted?  To  put  these  factors  into  an  applied 
military  context,  we  can  examine  the  steps  involved  in  a  group  planning  activity,  from  Klein  and 
Miller’s  (1996)  observation  of  a  Joint  Forces  Air  Component  Command  exercise.  They  show  these 
functions,  sources,  and  communications  as  being  the  steps  taken  during  the  production  of  air  combat 
orders. 


Problem  Detection 

•  Joint  Intelligence  Cell  provides  intelligence  to  many  groups,  including  the  Joint  Force  Air 
Component  Commander  (JFACC)  staff 

•  JFACC  intelligence  cells  feeds  intelligence  to  JFACC  staff 

•  Problems  are  identified  and  corrected  during  briefings  in  the  intelligence  cell  and  at  the  Joint 
Target  Coordination  Board 

•  Combat  Plans  and  Combat  Operations  detect  problems  as  plan  is  worked  out,  and 
implemented,  respectively 

•  Smaller  teams  within  the  JFACC  staff  detect  low-level  problems  such  as  flight  deconfliction 

Information  Gathering 

•  Information  gathered  by  intelligence  from  sensors  and  other  data  collecting  assets  such  as 
imagery,  gun  camera  video,  electronic  interception,  forward  observers,  etc. 

•  Members  of  Combat  Plans  include  experts  on  platforms  and  weapons  systems  who  can 
answer  questions 

Formulating  the  Course  of  Action 

•  Target  prioritizing  is  done  by  the  Intelligence  Cell  from  component  target  nominations 

•  Course  of  action  is  captured  in  the  Joint  Integrated  Prioritized  Target  List  (JIPTL),  Master  Air 
Attack  Plan  (MAAP),  and  finally  in  the  Air  Tasking  Orders  (ATO) 

•  Joint  Targeting  Cell  Board  (JTCB)  approves  and/or  modifies  the  course  of  action 

•  Detailing  the  Course  of  Action 

•  Staff  in  Combat  Plans,  with  input  from  the  wings,  detail  the  course  of  action  in  isolation  from 
the  staff  that  formulated  the  course  of  action 

Evaluating  the  Plan 

•  Corrections  made  as  the  plan  is  being  developed 

•  Evaluation  accomplished  in  meetings,  especially  by  the  JTCB 

•  No  formal  evaluation  mechanism 
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Problem  Representation 

•  Tactical  maps  and  overlays  at  each  level  of  planning 

•  Numerous  representations  that  differ  at  different  levels  of  planning,  including  Concept  of 
Operations  (CONOPS),  target  nominations,  briefings,  J1PTL,  MAAP,  ATO,  etc. 

•  Generation  and  transmission  of  Commander’s  intent  and  guidance  documents  (p.24) 

3.1.3.2  Distributed  vs.  Collocated 

We’ve  discussed  some  of  the  factors  affecting  group  decision-making  as  opposed  to  individual 
decision-making.  Now  we  need  to  address  the  distributive  nature  of  decision-making.  How  does  the 
distributive  component  affect  decision-making?  What  are  the  differences  between  collocated  groups 
and  geographically  dispersed  groups?  Obviously,  dispersed  groups  rely  on  communications 
technology,  which  certainly  affects  human  communication.  Support  for  collaborative  work  includes 
tools  such  as  electronic  mail,  bulletin  boards,  shared  information  systems,  video-conferencing,  and 
electronic  meeting  rooms.  These  technologies  allow  individuals  who  are  dispersed  in  time  and 
location  to  work  on  common  tasks.  This  area  has  been  supported  by  systems  known  as  groupware, 
group  decision  support  systems,  computer-supported  cooperative  work,  and  computer-supported 
collaborative  work  (Hiltz,  Dufner,  Fjermestad,  Kim,  Ocker,  Rana,  &  Turoff,  1996).  By  the  year 
2025,  it  is  likely  that  the  information  environment  will  incorporate  elements  of  virtual  reality. 
Networked,  distributed  VR  would  allow  immersive,  multi-participant  interaction.  University  of 
Washington’s  Human  Interface  Technology  Laboratory  demonstrated  a  common  virtual  world  called 
GreenSpace  for  users  in  Tokyo  and  Seattle,  via  a  narrow-band  communication  link.  Users  had  a 
representation  of  being  in  the  same  room  with  one  another.  Compared  to  teleconferencing,  VR  offers 
an  immersive,  3-D  experience. 

A  benefit  of  a  distributed  decision-making  team  is  that  members  may  be  positioned  in  ways  that 
extend  the  reach  of  a  collocated  team  (Orasanu  &  Salas,  1993).  Different  information  is  available 
through  the  visual  scene,  for  example,  when  members  of  a  group  are  not  located  together.  More 
computer  and  other  hardware  resources  may  be  available  when  a  group  is  not  collocated.  The 
downside  of  geographic  dispersion  is  that  communication  information  is  lost  when  communication  is 
not  face-to-face.  That  is,  facial  expressions,  volume,  and  emphasis  that  we  observe  when  we  are  in 
the  same  physical  space  as  another  person,  may  not  be  available  through  an  electronic  medium. 
Advances  in  virtual  reality  may  alleviate  some  of  this  information  loss.  The  geographical  separation 
also  places  another  burden  on  the  team,  that  of  differentiating  tasks  and  distributing  them.  In  other 
words,  the  management  load  increases  when  the  team  is  dispersed. 

3.2  EFFECTIVE  DECISION-MAKING  GROUPS  -  ENVIRONMENT  AND  COMPOSITION, 
INDIVIDUAL  DIFFERENCES,  AND  SKILLS 

The  Air  Force  needs  to  know  how  to  prepare  for  the  next  century.  We  acknowledge  that  the  future 
battlefield  will  contain  as  a  crucial  element,  distributed  collaborative  decision-making.  How  the  Air 
Force  can  select  or  train  good  decision-makers  is  the  focus  of  this  section,  which  details  some  of  the 
research  regarding  the  influences  on  group  function.  What  can  we  extract  from  the  scientific 
literature  about  effective  teams?  Studies  are  full  of  indicators  regarding  innate  human  characteristics, 
cognitive  abilities,  factors  external  to  the  group,  group  processes,  and  ways  of  behaving.  What  sense 
can  we  bring  to  these  data?  What  characteristics  of  effective  decision-making  teams  can  we  use  in 
Air  Force  selection  and  training  policies?  This  section  highlights  some  of  the  ongoing  research 
programs  into  training  methods  for  effective  decision-making.  Literature  was  chosen  that  focused  on 
either  personality  traits  of  effective  decision-makers  or  on  strategies  or  ways  of  behaving  in  a  group, 
that  could  be  trained. 
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This  Review  &  Analysis  has  discussed  some  of  the  issues  involved  in  the  study  of  collaborative 
distributed  decision-making.  At  this  point,  we  need  to  extract  some  information  that  is  useful  to  the 
Air  Force  in  its  strategic  planning  for  the  next  decades.  The  tack  will  switch  now  to  effective 
decision-making,  and  how  an  effective  decision-making  unit  can  be  generated.  This  section  will 
focus  on  three  main  topics:  (1)  the  group’s  make-up  and  environment,  (2)  individual  abilities  and 
traits,  and  (3)  decision-making  strategies  that  characterize  good  decision-makers.  As  was  mentioned 
earlier,  some  of  these  variables  are  totally  under  the  control  of  the  Air  Force;  others  are  inherent  to  the 
individual  and  would  be  important  to  Air  Force  selection  processes;  while  other  factors  are  trainable. 

3.2.1  Effective  Decision-Making  Groups — The  Environment  and  Group  Composition 

This  section  describes  some  factors  that  the  Air  Force  can  impose  upon  group  processes.  These 
variables  are  external  to  the  individual,  that  is,  the  Air  Force  can  determine  for  example,  the 
composition  of  the  group,  and  the  resources  available  to  the  group.  The  work  of  Levine  and 
Moreland  (cited  in  Kokorian,  1995)  provides  a  good  beginning  to  this  discussion.  Their  work 
summarizes  the  factors  that  influence  decision-making  groups,  including  the  environment,  individual 
traits,  and  strategies.  Since  their  work  emphasizes,  but  extends  beyond,  environmental  issues,  it  will 
serve  as  a  good  introduction  to  the  topic  of  factors  that  engender  good  decision-making.  Levine  and 
Moreland  provide  this  list  of  elements  that  affect  group  behavior.  They  use  the  word  “ecology”  as  it 
refers  to  the  physical,  social,  and  temporal  environment  in  which  the  group  works.  The  context  of 
group  behavior  needs  to  be  taken  into  account.  Context  items  include  the  following: 

Physical  Environment  —  the  impact  of  computerization  on  changes  to  group 
communication,  norms  and  status  systems,  and  factors  influencing  a  sense  of 
group  territoriality  and  shared  ownership  over  physical  and  temporal  space. 

(P-  4) 

Social  Environment  —  a  move  towards  models  based  on  intra-group 
collaboration  through  such  processes  as  negotiation,  acquisition  of 
information,  impression  management  and  buffering  (defense),  research  on 
the  influence  of  culture  on  groups  and  groups  on  culture,  (p.  4) 

Temporal  Environment  —  which  refers  both  to  the  development  level  of  the 
group  (new  or  mature)  and  to  the  influence  of  time  limits  and  deadlines  on 
group  behavior,  (p.  4,  5). 

Levine  and  Moreland  also  state  that  group  composition  is  an  important  determinant  of  group 
behavior.  Important  variables  that  influence  group  interactions  include  group  size,  and  demographic, 
affective,  and  skills  characteristics  of  a  group.  Salience  between  characteristics  of  group  members  is 
also  important,  as  is  mode  of  conflict  resolution.  Also  to  be  considered  is  the  pattern  of  relationships 
among  group  members. 

Many  of  these  environmental  variables  could  be  controlled  externally,  that  is,  by  Air  Force 
assignments  of  individuals  to  groups,  or  the  provision  of  technology,  and  the  nature  of  the  constraints 
and  freedoms  extended  to  the  group. 

3.2.2  Effective  Decision-Making  Groups:  Individual  Differences 

Selection  of  personnel  by  Air  Force  planners  is  the  issue  with  variables  of  this  type.  It  is  important 
for  Air  Force  planners  and  strategists  to  consider  ways  in  which  personnel  selection  can  be  modified 
in  future  decades  so  that  Air  Force  personnel  will  be  effective  decision-makers  as  well  as  competent 
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in  their  technical  area.  This  section  reviews  some  of  the  skills,  abilities,  and  characteristics  of  people 
who  will  contribute  to  effective  team  decision-making.  The  drawback  to  this  area  of  research  is  that 
much  of  it  is  situation-specific.  There  appears  to  be  no  consensus  about  what  innate  variables  predict 
good  decision-making. 

Individual  differences,  including  abilities  and  personality  traits,  have  been  considered  by  many 
researchers  as  a  means  to  predict  team  or  group  performance.  There  is  general  consensus  among 
researchers  that  individual  difference  variables,  including  personality,  can  have  a  significant  influence 
on  group  functioning  (Hogan,  Raza,  Sampson,  Miller,  &  Salas,  1989;  Kichuk,  1996;  Hare,  1962). 
While  this  notion  is  generally  supported,  the  exact  nature  of  the  impact  of  personality  on  groups  is  not 
easy  to  quantify. 

Larson  and  LaFasto  (cited  in  Kokorian,  1995)  cite  certain  factors  as  important  to  membership  of  an 
effective  decision-making  team.  While  some  of  the  factors  they  cite  are  skill-based  (the  subject  of  the 
next  section),  others  are  considered  inherent  to  the  individual.  For  instance,  Larson  and  LaFasto 
indicate  personal  competencies  such  as  intellectual  ability,  results  orientation,  team  orientation, 
maturity,  and  personal  presence  are  important,  as  are  a  strong  desire  to  contribute  and  the  ability 
collaborate  effectively. 

Hogan  et  al.  (1989)  conducted  a  two  and  one-half  year  project  for  the  Navy  investigating  the  impact 
of  personality  on  team  performance.  They  concluded  that  “the  personality  characteristics  associated 
with  optimal  team  performance  depend  on  the  type  of  task”  (p.  3).  Specifically,  their  research 
supported  the  following  personality  and  task  combinations  for  optimal  performance:  teams  composed 
of 


•  reserved,  conscientiousness  rule-followers  perform  best  on  realistic  tasks. 

•  extroverted,  independent  risk  takers  perform  best  on  investigative  tasks. 

•  daring  individuals  with  broad  interests  perform  best  on  artistic  tasks. 

•  conscientious,  planful,  affiliative  people  perform  best  on  social  tasks. 

•  intellectually  focused ,  emotionally  well-adjusted,  independent  people  who  are  not 
personally  ambitious  perform  best  on  enterprising  task?,. 

•  reserved,  conscientious  rule  followers  who  are  intellectually  motivated  perform  best  on 
conventional  tasks,  (p.  6) 

In  other  studies,  the  Myers-Briggs  Type  Indicator  (MBTI)  was  used  to  measure  the  correlate  between 
personality  and  group  performance.  One  such  project  examined  the  decision-making  ability  of 
groups  formed  with  dissimilar  personality  types  and  those  not  formed  by  dissimilar  types.  The 
research  concluded  that  the  teams  formed  by  dissimilar  MBTI  personality  types  did  score  higher  than 
teams  formed  without  the  MBTI,  but  they  were  statistically  no  more  effective  (Page,  1991).  Another 
study  examined  the  relationship  between  team  members’  MBTI  scores  and  the  teams’  subsequent 
performance.  The  results  indicate  that  “low  heterogeneity  among  the  team  members’  scores  on  the 
factor  of  extraversion  was  found  to  correlate  positively  and  significantly  to  the  team’s  actual 
performance”  (Kichuk,  1996,  p.  1).  Further,  it  was  noted  that  “heterogeneity  among  the  team 
members’  scores  on  the  factor  emotional  stability  was  found  to  be  positively  related  to  the  actual 
performance  of  the  team”  (Kichuk,  1996,  p.  1).  In  addition,  she  states  that  while  one  slightly  neurotic 
person  (low  on  the  emotional  stability  factor)  may  be  beneficial  to  team  performance,  more  than  one 
in  a  small  group  is  detrimental  to  team  performance. 

Hare  (1962)  discusses  the  effects  of  personality  combinations  in  small  group  interactions.  In  general, 
he  states  that  “a  high  group  mean  on  a  personality  trait  usually  results  in  a  similar  type  of  behavior  for 
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the  group”  (p.  195).  Also,  “high  variance  on  other  traits  may  produce  more  effective  groups  if  the 
task  calls  for  variability  in  individual  performance,  or  conflict  if  the  task  calls  for  similarity  of 
individual  performance”  (p.  195).  In  addition,  he  makes  the  following  group  distinctions: 

•  High  group  means  ...  [for]  adventuresomeness ,  vigor,  dominance,  purposefulness, 
orderliness,  willed  application,  and  freedom  from  anxiety  are  associated  with  ...  high 
performance  on  tasks  requiring  vigorous  coordinated  action. 

•  High  group  means  . . .  [for]  paranoid  suspiciousness,  nervous  tension,  emotional 
immaturity,  worrying  suspicious  anxiety,  and  lack  of  self-sufficiency  are  associated 
with  low  observer  ratings  on  degree  of  leadership,  orderliness,  we-feeling,  level  of 
motivation,  degree  of  group  organization  and  interdependence,  and  with  high  ratings 
on  frustration. 

•  High  variances  among  members  on  personality  traits  of  surgency  ...,  radicalism  ..., 
and  high  variance  on  degree  of  internalization  of  social  norms,  along  with  high  mean 
friendliness,  intelligence,  and  general  level  of  radicalism ,  are  associated  with  a  high 
level  of  accuracy  on  tasks  requiring  a  judgment  of facts  from  inferential  grounds. 

•  High  variances  on  personality  measures  of  tough-  versus  tender-mindedness, 

(i  Bohemian  aggressiveness,  ”  and  paranoid  suspiciousness  are  associated  with  dislike 
for  a  task  of  resolving  opinion  and  attitude  differences,  slowness  in  ranking  attitude 
preferences,  and  a  feeling  by  members  that  other  members  hinder  group  progress. 

•  Low  variance  or  uniformity  [of  emotional  maturity]  is  found  to  be  associated  with  a 
certain  optimism  and  confidence  in  level  of  aspiration. 

•  Measured  personality  traits  of  members  involving  emotional  maturity,  friendly  co¬ 
operativeness,  trustfulness,  and  adaptability,  adventuresomeness,  willed  application, 
and  freedom  from  anxiety  were  positively  related  to  smooth  and  effective  group 
functioning. 

•  The  characteristics  of  paranoid  suspiciousness,  eccentricity,  and  coolness  to  others 
were  found  to  be  negatively  related  to  smooth  functioning,  (p.  191-3) 

Studies  have  also  been  conducted  in  Air  Force  situations  to  examine  the  utility  of  personality 
measures  and  their  corresponding  evaluative  tests  for  aircrew  selection.  Pedersen,  Allan,  Laue, 
Johnson,  and  Siem  (1992)  conclude  that  the  Myers-Briggs  Type  Indicator  is  the  most  viable  option  to 
guide  future  AF  research  in  the  selection  of  personnel  based  on  personality.  The  Five-Factor  Model 
“provides  a  structural  description  of  the  basic  dimensions  of  personality”  (< extraversion ; 
agreeableness;  conscientiousness;  emotional  stability;  intellect %  and  thus  “serves  as  a  scientific 
framework  for  the  development  of  predictor  constructs  for  selection”  (p.  16).  However,  Dolgin  and 
Gibb’s  (1989)  review  of  personality  literature  in  relation  to  aircrew  selection  revealed  that  most 
personality  tests  do  not  produce  valid  measures  for  pilot  selection.  They  do  however  recommend 
continued  research  of  the  Defense  Mechanism  test,  Personality  Research  form,  and  the  Strong 
Vocational  Interest  Blank  as  they  appear  to  be  both  effective  pilot  selection  tools  and  soundly 
designed  measures.  Nonetheless,  “there  is  still  doubt  as  to  the  predictive  ability  of  any  personality 
measure  for  predicting  aircrew  performance  (Pedersen  et  al.,  1992,  p.  23). 
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The  information  about  aptitudes  and  personality  traits,  and  how  to  combine  these  variables  into  an 
optimum  mix  of  individuals,  represents  some  interesting  findings.  Yet  much  of  this  research  results 
in  findings  that  are  situation-specific.  A  strong  answer  about  the  inherent  characteristics  of  a  good 
group  decision-maker  does  not  yet  exist. 

3.2.3  Effective  Decision-Making  Groups  -  Skills  and  Strategies  through  Experience 

This  section  has  reported  on  some  of  the  variables  that  affect  group  processes.  Importantly,  this 
review  of  the  literature  revealed  that  while  some  studies  focus  on  personality  characteristics  of  good 
decision-makers,  or  the  effects  of  group  composition  or  environment,  a  great  many  studies  focus, 
instead,  on  what  can  be  done  to  any  set  of  individuals  to  turn  them  into  good  group  decision-makers. 
There  is  a  plethora  of  evidence  that  learning  is  the  crucial  element  for  good  group  decision-making — 
that  expert  decision-makers  are  experts  because  they  are  experienced  decision-makers.  What  follow 
are  three  studies  that  describe  the  capabilities  and  strategies  of  experienced  group  decision-makers. 
There  is  consensus  about  what  makes  good  decision-makers  or  a  good  decision-making  teams — 
experience.  Experienced  decision-makers  are  expert  decision-makers.  This  finding  has  major 
implications  for  Air  Force  personnel  and  training  programs.  This  is  one  of  the  most  important 
sections  of  this  Review  &  Analysis.  The  findings  here  represent  some  of  the  best  research  performed 
in  the  area  of  group  decision-making,  and  the  results  of  all  the  research  are  more  consistent  in  this 
area  of  strategies  than  in  the  areas  of  individual  differences  or  environment. 

Orasanu  (cited  in  Kokorian,  1995)  presents  these  characteristic  ways  in  which  effective  teams  behave. 
Expert  decision-making  teams  are  characterized  by  the  following: 

•  Situational  Awareness — that  crews  are  alert  to  developing  situations,  sensitive  to 
clues,  and  aware  of  their  implications 

•  Planf illness — that  crews  work  out  plans  and  strategies  for  reaching  their  goals  in 
advance,  that  they  prepare  for  contingencies,  determine  what  information  they 
need,  and  monitor  their  progress 

•  Shared  Mental  Models — that  crews  communicate  efficiently  enough  to  create  a 
shared  big  picture  through  which  they  use  resources,  make  sure  they  are  all 
solving  the  same  problem,  and  solve  it  in  a  coordinated  way 

•  Resource  Management — that  groups  use  resources  efficiently  and  explicitly  by 
setting  priorities,  scheduling  tasks,  allocating  responsibilities,  and  including 
thinking  time  (p.  33) 

Kanki,  Lozito,  and  Foushee  (1989)  state  that  experienced  teams  can  become  so  efficient  in  their  work 
that  they  are  able  to  overcome  effects  of  fatigue.  They  describe  an  experienced  team  as 

•  sharing  a  mental  model 

•  using  conventionalized  conversation  pattern 

•  using  task-relevant  talk 
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Gordon  (1992)  describes  an  experienced  decision-maker  as 

•  having  better  organized  knowledge  structures 

•  perceiving  and  organizing  on  an  abstract  level 

•  perceiving  problems  in  meaningful  patterns 

•  using  procedural  knowledge  and  forward  inferencing  techniques 

Shanteau  (1988)  states  that  an  experienced  decision-maker 

•  can  discern  relevant  from  irrelevant  information 

•  can  simplify  complexities 

•  knows  when  to  make  exceptions 

•  can  choose  appropriate  problems  to  solve 

•  has  content  area  knowledge 

•  has  cognitive  automaticity 

While  the  earlier  sections  reported  findings  about  skills,  abilities,  personality,  and  group  composition, 
those  findings  were  disparate  from  one  another  and  hard  to  generalize  to  many  kinds  of  decision¬ 
making  environments.  However,  the  work  in  this  current  section  shows  much  more  consensus  and 
generalizability.  These  studies  all  concur  on  the  benefits  of  training,  of  being  an  experienced 
decision-maker.  As  Air  Force  strategists  plan  for  the  next  century,  the  implications  for  training 
cannot  be  overemphasized. 


3.3  COLLABORATIVE  DISTRIBUTED  DECISION-MAKING  RESEARCH  PROGRAM- 
REPRESENTATIVE  EFFORTS  FROM  AIR  FORCE,  ARMY,  NAVY,  AND  OTHER 
ARENAS 

This  section  reviews  programs  and  research  on  collaborative  distributed  decision-making,  largely 
from  the  military  arena.  The  first  area  examined,  Current  DoD  Distributed  Collaborative  Decision- 
Making  Programs,  focuses  on  uses  of  this  process  already  in  effect  throughout  the  Department  of 
Defense.  Subsequent  areas  focus  on  research  programs. 

3.3.1  Current  DoD  Distributed  Collaborative  Decision-Making  Programs 

Many  of  the  DoD  futuristic  and  planning  documents  include  functions  and  processes  that  presume 
collaborative  decision-making.  Recognizing  how  crucial  the  collaborative  nature  of  future  decision¬ 
making  is,  several  military  programs  are  emphasizing  this  emerging  research  area.  Army,  Navy,  and 
Air  Force  are  all  supporting  collaborative  distributed  decision-making  research,  results  of  which  will 
be  discussed  later.  The  following  are  just  a  few  of  the  decision-making  efforts  in  a  variety  of  arenas, 
to  show  the  diversity  of  interest  in  collaborative  decision-making. 

In  May  of  1996,  a  DOD  Collaborative  Technology  Users  Network  forum  was  held  with 
representatives  from  the  HQ  USMC  staff,  HQ  USAF  Innovation  Center,  the  Defense  Information 
Systems  Agency,  National  Defense  University,  US  Army  Community  Family  Support  Center,  US 
Navy  Office  of  Civilian  Personnel  Management,  and  the  US  Navy  Manpower  and  Personnel  office. 
The  purpose  of  this  forum  was  to  share  information  on  creating  joint  models  on  how  to  make  better 
use  of  collaborative  meeting  technology  throughout  the  DOD.  DARPA’s  Joint  Task  Force  Advanced 
Technology  program  developed  an  “anchor  desk”  as  a  prototype  to  support  a  forward-deployed  Joint 
Force  commander  with  technologies  such  as  computer-supported  cooperative  work  technology  and 
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hypermedia  technology.  Notably,  the  Information  Resources  Management  College  offers  a  course  to 
OS/GM  12-15  and  military  grades  0-4  and  0-6  on  Group  Decision-Making  and  Groupware.  Also,  a 
recent  colloquium  sponsored  by  the  Human  Factors  and  Ergonomics  Society  was  titled  Multi-Crew 
Performance  in  Complex  Military  Systems.  The  intent  of  this  colloquium  was  to  establish  research 
approaches  to  complex  military  systems  with  a  crew-centered  perspective,  using  the  various 
disciplines  of  human-computer  interaction,  computer-supported  cooperative  work,  NDM,  multi-crew 
aiding,  cognitive  engineering,  situated  cognition,  and  cooperative  learning.  Much  of  what  was 
presented  was  prospective,  or  work  in  progress,  but  later  we  can  describe  a  few  emerging  possibilities 
as  an  introduction  to  what  we  know  about  group  decision-making  and  performance. 

Research  funded  by  the  Air  Force,  Army,  and  Navy  focuses  on  a  wide  range  of  issues.  Some  research 
is  applied  (the  work  of  Klein  for  instance,  which  has  resulted  in  a  training  program),  whereas  other 
programs  are  still  theoretical,  seeking  to  establish  constructs  and  definitions,  and  to  produce  valid  and 
reliable  tools  to  measure  whether  a  group  is  working  effectively  or  not.  Much  of  the  work  is  directly 
relevant  to  the  design  of  programs  that  train  groups  in  effective  decision-making.  Although  much  is 
known  about  decision-making,  at  this  stage,  little  is  known  about  how  to  train  decision-making  for 
complex  environments  (Salas  &  Cannon-Bowers,  1996).  Many  current  researchers  in  the  field  echo 
this  conclusion.  Kirlik,  Walker,  Fisk,  and  Nagel  (1996,  p.  289)  state,  about  decision-making  in 
complex  environments,  . .  little  is  known  about  how  best  to  train.  . .  this  component  of  skilled 
behavior.”  Many  of  the  issues  brought  up  at  the  Human  Factors  and  Ergonomics  Society  colloquium 
on  Multi-Crew  Performance  in  Complex  Military  Systems  focused  on  defining  relevant  concepts  and 
determining  ways  of  measuring  effective  group  performance,  concerns  which  seem  quite  preliminary. 
While  these  endeavors  seem  to  be  in  their  early  stages  of  development,  there  is  much  to  be 
encouraged  about.  Many  training  programs  are  under  development,  and  a  great  deal  of  research  is 
being  done  on  how  to  design  an  effective  training  program. 

3.3.2  Current  DoD  Distributed  Collaborative  Decision-Making  Research 

While  some  work  is  being  done  in  the  military  arena  that  focuses  on  traits  and  abilities  (see  Hogan,  et 
al.,  1989,  for  example),  the  bulk  of  the  decision-making  research  focuses  on  skills  training.  The  goals 
of  these  research  programs  are  varied.  The  work  of  Klein  and  associates  is  applied  in  thrust,  while 
the  work  of  McNeese  is  very  laboratory-based.  This  brief  view  of  the  many  active  programs  shows 
the  wide  range  of  goals  and  focuses. 

3.3.2.1  Modeling  Army  C2  at  the  Brigade  Level 

Ensing  and  Knapp  (1995)  describe  a  methodology  for  modeling  tasks  and  workload  for  optimal 
allocation  of  personnel  resources  in  a  Command  and  Control  (C2)  Center.  The  approach  is  process- 
oriented,  with  the  goal  of  decomposing  cognition,  decision,  and  group  tasks.  They  make  the  point 
that  while  functional  analyses  and  training  manuals  exist  for  many  C2 centers,  process  analyses  are 
marginal. 

A  functional  analysis  or  training  document  lists  tasks  such  as  “analyze”  and  “decide,”  without 
elaborating  how  these  are  performed.  The  Functional  Description  Documents  are  usually  at 
too  high  a  level  and  fail  to  account  for  the  natural  sequence  of  functions.  Training  manuals 
contained  sequences  but  are  confined  to  manual  operations.  Neither  contains  process  steps 
for  how  to  “analyze”  or  “decide”  (p.  2). 

This  study  decomposed  these  types  of  tasks  into  meaningful  steps.  Subject  matter  experts  knew  their 
end  product — analysis  or  decision — but  could  not  articulate  how  they  got  there.  Facilitation  of  their 
mental  steps  took  place,  with  such  suggestions  as  “look  at  the  ammo  level  on  the  board,”  “compare  it 
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with  the  ammo  level  on  the  plan.”  From  the  information  thus  extracted,  the  cognitive  skills  and 
abilities  involved  in  analysis  and  deciding  were  derived. 

A  baseline  work  flow  based  on  a  Maneuver  Brigade  Tactical  Operations  Center  was  the  basis  for  this 
model.  A  tiered  approach  was  used  to  form  an  analysis  framework.  In  the  first  tier,  the  work  flow 
and  mental  processing  demands  were  established.  Each  subsequent  tier  examined  variations  for 
workload  comparisons.  The  second  tier  posed  the  question,  “What  effect  would  software  have  on 
staff  processes,  the  cognitive  skills  required,  the  type  of  decisions  imposed  by  the  software,  number 
of  people  required,  and  how  they  interact?”  From  these  data,  the  third  tier  adds  the  effect  of 
environmental  stress  factors,  such  as  heat,  noise,  and  vibration.  For  instance,  the  effect  of  noise  on 
group  interaction  might  be  examined.  Tier  four  examines  the  effects  of  removing  face-to-face 
interaction  on  group  processes.  The  fifth  tier  examines  imperfect  communications  such  as  the  effect 
of  data  exchange  rates  and  limited  voice  communication.  Using  this  tiered  approach  allows  the 
researchers  to  isolate  the  effects  of  specific  variables.  Data  are  being  gathered  at  this  time  to  assess 
the  effects  of  the  different  variables  on  C2  group  performance. 

3.3.2.2  Advanced  Team  Decision-Making 

Much  of  the  work  on  designing  training  programs  for  effective  group  decision-making  has  been 
accomplished  by  Zsambok,  Klein,  Kyne,  and  Klinger  (1993).  These  researchers  based  their  training 
program  principles  on  their  observations  of  expert  and  inexpert  decision-making  groups.  Groups  from 
these  organizations  were  studied  or  trained: 

•  Blue  Flag  at  Hurlburt  Field,  FL 

•  AEGIS  Combat  Information  Center  at  the  Combat  System  Engineering  Development  Site 
(CSEDS),  Moorestown,  NJ 

•  Corps-level  exercises  at  the  US  Army  War  College  (AWC) 

•  Corps  and  division  exercises  at  the  US  Army  Command  and  General  Staff  College  (USACGSC) 

•  Echelons  above  corps  at  the  National  Defense  University  (NDU) 

•  Brigade  and  battalion  exercises  at  Fort  Hood,  TX;  Fort  Stewart,  GA;  and  the  National  Training 
Center,  Fort  Irwin,  CA 

•  Logistics  teams  at  the  Air  Force  Institute  of  Technology,  Wright- Patterson  Air  Force  Base,  OH 

•  Helicopter  teams  at  Fort  Campbell,  KY 

•  Commercial  aviation  crews  at  the  NASA  Ames  Research  Center,  Moffett  Field,  CA 

Zsambok  et  al.  (1993)  found  these  commonalties  among  effective  teams,  and  produced  a  training 
programs  upon  these  observations.  The  model  of  advanced  team  decision-making  based  on  these 
observations  includes  these  three  basic  concepts: 

Team  Identity  is  the  extent  to  which  a  team  sees  itself  as  a  unit  that  is  interdependent,  and  uses  that 
perspective  during  team  decision-making.  It  is  concerned  with 

•  members’  understanding  of  one  another’s  roles  and  functions,  how  well  they  perform  them,  and 
the  ability  of  the  team  to  uncover  hidden  expertise 

•  the  level  of  involvement  of  each  member  in  the  task  and  the  ability  of  the  team  to  engage  all 
members 

•  the  ability  of  the  team  members  to  temporarily  leave  their  role  or  function  and  compensate  for 
gaps  in  team  performance 

•  the  ability  of  the  team  to  avoid  micromanagement 
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Team  Conceptual  Level  represents  the  team  as  an  intelligent  entity.  It  includes  the  group’s  ability  to 
think,  solve  problems,  and  make  decisions  collectively.  It  also  refers  to  the  information  explicitly 
known  by  all  team  members.  It  includes  the  abilities  of  teams  to: 

•  envision  goals  and  process  plans  to  reach  those  goals,  initially  and  iteratively 

•  focus  appropriately  on  their  time  line,  and  the  range  of  factors 

•  seek  out  gaps  and  ambiguities  in  their  information  base  or  situation  assessment 

•  seek  out  divergent  situation  assessments  before  deciding  on  a  single  one 

Team  Self-Monitoring  is  really  metacognition,  and  includes  monitoring  and  regulating  team 
performance,  on  such  variables  as  ability  to  allocate,  monitor,  and  re-prioritize. 

Zsambok  et  al.  (1993)  believe  that  teams  can  learn  to  work  effectively,  that  it  is  possible  to  train 
teams  to  behave  in  the  ways  just  outlined.  Being  a  good  decision-maker  is  a  skill  that  can  be  taught. 

Klein  (1993)  and  Cannon-Bowers,  Tannenbaum,  Salas,  &  Volpe  (1995)  have  shown  that  teamwork 
skills  are  separate  from  work  skills  and  they  require,  and  benefit  from,  dedicated  training.  What  is  it 
that  teams  can  learn  to  do?  After  training,  teams  can: 

•  adjust  their  responses  in  accordance  with  task  demands  (stress) 

•  employ  implicit  coordination  strategies  without  overt  communication  by  drawing  upon  common 
knowledge  bases 

•  build  complex  associations  between  situational  cues  and  appropriate  responses  and  strategies 

Klein,  Zsambok,  and  Thordsen  (1993)  believe  strongly  in  training  designed  specifically  for  team 
work,  and  that  teampower  is  the  resulting  strength.  The  basic  tenets  of  their  training  program  focus 
on  the  following  standards: 

•  Task  exercises  do  not  necessarily  provide  team  training 

•  Instructors  must  learn  how  to  train  teamwork  skills 

•  Team  decision  training  can  be  added  existing  training 

•  Team  decision  skills  can  be  reliably  measured 

•  Participants  can  be  taught  to  monitor  their  team  in  action  and  to  make  adjustments 

•  Teampower  is  important  and  it  can  be  developed,  (p.  42) 

Others  agree  that  training  is  essential  to  having  an  effective  decision-making  group.  In  a  technical 
report  on  expert  pilot  decision-making,  Adams  and  Lofaro  (1992)  state  that  the  primary  differences 
between  being  a  beginner  and  being  an  expert  can  be  the  acquisition  of  knowledge  and  problem¬ 
solving  skills.  Expert  performance  can  be  defined  as  coming  up  with  an  appropriate  response  to 
problems  or  situations.  The  novice  often  solves  problems  by  weak,  domain-general  methods,  often 
working  backwards  from  the  goal.  With  experience  or  training,  domain-specific  production  rules  can 
be  quickly  developed.  We  know  now  that  experienced  decision-makers  are  expert  decision-makers, 
so  the  question  becomes,  how  can  we  turn  naive  decision-makers  into  experienced  decision-makers? 
Clearly,  good  decision-makers  are  made  and  not  bom. 

In  answer  to  the  implied  question,  why  train  for  teamwork,  Klein  et  al.  (1993)  reply: 

Many  teams  are  not  together  very  long,  and  do  not  have  much  time  to  come  up  to  speed. 

That  means  the  members  have  to  be  effective  team  decision-makers  before  they  join  the  team. 
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The  only  way  for  that  to  happen  is  for  them  to  learn  the  necessary  team  skills  beforehand — 
through  team  training  and  prior  experiences,  (p.  42) 

3.3.2.3  Air  Force — Living  Lab 

One  of  the  Air  Force’s  collaborative  distributed  processes  projects  seems  to  have  taken  a  step  back,  to 
a  more  theoretical  examination  of  the  factors  involved  in  group  work.  The  work  of  McNeese  and 
others  focuses  on  a  more  academic  approach  that  seeks  to  investigate  and  define  the  mechanics  of 
how  people  in  groups  work  together.  This  way  of  dealing  with  decision-making  training  may  prove 
to  be  a  good  choice,  since  its  integrated  approach  of  research  and  operations  concerns  may  generate 
some  solid  training  principles.  Future  work  in  the  Living  Lab  will  focus  on  natural  modes  of 
interaction  among  remote  and  collocated  sites.  McNeese  (1996a)  states: 

Cognitive  science  concerns  for  USAF  new  millennium  initiatives  revolve 
around  potentially  devastating  requirements  for  attention,  memory,  group 
problem-solving,  perception,  learning,  knowledge  acquisition  access,  and 
motor  control.  These  concerns  may  be  jointly  referred  to  as  distributed 
cognition.  A  more  theoretical  approach  is  seen  in  the  operations  of  the 
Living  Lab,  implemented  at  the  Paul  M.  Fitts  Human  Engineering 
Laboratory  of  the  Air  Force’s  Armstrong  Laboratory.  While  attempting  to 
integrate  theory  and  practice,  the  lab  seeks  to  include  the  scientific  research 
base  community  and  the  operational  context/  warfighter  community,  (p. 

264) 

McNeese  (1996b),  in  a  description  of  the  Living  Lab,  states  that  much  of  today’s  work,  in  many 
arenas,  including  the  military,  government,  or  private  industry,  is  done  in  teams.  Teams  may  contain 
smaller  units  referred  to  as  multi-operator  enclaves.  Individuals  who  form  one  team  may  be  members 
of  other  teams  or  enclaves  that  consume  time  and  resources,  and  forge  relationships  in  layers  of 
complexity  and  change.  The  Living  Lab  is  based  on  the  belief  that  the  social-organizational, 
psychological-cognitive,  and  technological  components  of  collaboration  should  be  included  in  the 
study  of  group-distributed  decision-making.  It  is  imperative  that  a  broad  systems  approach  be  taken 
when  a  complex  system  of  computer-supported  collaborative  work  is  being  examined.  One  concept 
defined  by  Young  and  McNeese  (cited  in  McNeese,  1996b)  is  situated  cognition,  as  real-world 
problem-solving  “wherein  group  members  spontaneously  generate  knowledge  in  the  context  of  a 
situation;  coordinate  multiple  cognitive  processes,  applied  though  multiple  paths;  and  pick  up  critical 
perceptual  cues  for  potential  solutions”  (p.768).  The  framework  that  serves  to  organize  the  work  of 
the  Living  Lab  focuses  on  the  interactions  that  are  operative  in  any  collaborative  situation.  Three 
paradigms  are  used  to  study  collaborative  processes: 

3.3.2.3.1  Situated  and  Observed  Cognition  Studies  (SOCS)  program  that  seeks  to  understand 
collaborative  practices  within  real-world  work  environments.  Both  practices  and  tools  are 
highlighted. 

3.3.2.3.2  Tradeoffs,  Research,  and  Analysis  in  Collaborative  Ergonomics  (TRACE)  project  that 
examines  tradeoffs  as  they  occur  in  collaborating  design  teams. 

3.3.2.3.3  JASPER  (The  Adventures  of  Jasper  Woodbury:  Rescue  at  Boone’s  Meadow)  is  a 
cooperative  analogical  problem-solving  paradigm  that  investigates  the  roles  of  metacognition  and 
perceptual  expertise  in  collaborative  settings.  An  example  is  a  search  and  rescue  planning  task 
involving  an  ultralight  airplane.  One  outcome  of  such  an  investigation  is  to  determine  the  conditions 
within  group  collaboration  that  lead  to  a  group  member’s  use  of  knowledge  as  an  individual. 
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3.3.2.4  Air  Forc^-C3  STARS 


A  more  applied  Armstrong  Laboratory  program  is  called  C3  STARS — C  Simulation,  Training  and 
Research  systems.  Elliott,  Neville,  and  Dalrymple  (1996)  focus  on  factors  relating  to  Airborne 
Warning  and  Control  System  (AWACS)  Weapons  Director  team  performance  as  Weapons  Directors 
participate  in  an  air  campaign  scenario  that  includes  time  pressure,  high  complexity,  and  ambiguity. 
Imbedded  ambiguous  events  allow  the  monitoring  of  team  communications  and  decision-making. 
Communication,  coordination,  and  situational  awareness  are  assessed  relative  to  performance  and 
mission  accomplishment,  as  predicted  by  the  multi-level  theory  of  decision-making  in  hierarchical 
teams  with  distributed  expertise  (Hollenbeck,  Ilgen,  Sego,  Hedlund,  Major,  &  Phillips,  1995). 

The  effectiveness  of  AWACS  teams  depends  on  effective  communication,  coordination,  and  accurate 
situational  awareness.  Team  members  must  perform  information  transfer  activities  under  the  “fog 
and  friction”  of  war,  with  time  pressure  and  ambiguity,  which  may  result  in  “ineffective 
communications,  poor  situational  awareness,  ineffective  resource  allocation,  and  friendly  fire 
incidents”  (Elliott  et  al.,  1996,  p.  4).  There  may  be  missing  or  incorrect  information.  How  does  a 
good  team  maintain  communication  and  situational  awareness  under  such  circumstances?  These  are 
some  of  the  issues  C3  STARS  attempts  to  answer. 

To  create  a  realistic  distributed  scenario,  responsibilities  of  a  typical  air  campaign  are  assigned  to  the 
High  Value  Asset  Weapons  Director,  a  Combat  Air  Patrol  Weapons  Director,  and  the  STRIKE 
Weapons  Director.  The  High  Value  Assets  Weapons  Director  controls  the  C3  aircraft,  including  air 
refueling,  electronic  warfare  operations,  and  reconnaissance.  The  Combat  Air  Patrol  Weapons 
Director  controls  the  Defensive  Counter  Air  aircraft,  and  coordinates  the  fire  of  friendly  surface-to-air 
missile  assets,  and  team  leadership.  The  planned  bombing  missions,  unplanned  Suppression  of 
Enemy  Air  Defense  missions,  as 

well  as  the  unplanned  Theater  Missile  Defense  bombing  missions  are  controlled  by  the  STRIKE 
Weapons  Director.  Variables  under  investigation  by  this  lab  include 

•  team-level  measures  of  operational  outcomes 

•  team  outcome  measures 

•  preservation  of  assets 

•  destruction  of  enemy  assets 

•  team  efficiency/effectiveness  measures 

•  kill  ratio 

•  air  refuelings  completed 

•  completed  switch  actions 

•  friendly  aircraft  lost  of  fuel  depletion 

•  fratricide 

•  penetration 

•  situational  awareness  measures 

•  total  number  of  times  assigned  symbology  (airbome/downed  tracks)  becomes  uncorrelated 

Data  are  currently  being  gathered  and  early  results  indicate  that  the  measures  used  are  reliable  ones. 
Future  analysis  will  concentrate  on  effects  of  information  ambiguity,  and  what  communication  and 
coordination  strategies  are  used  by  teams  who  deal  effectively  with  ambiguity. 
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3.3.2.5  Navy — Tactical  Decision-Making  Under  Stress  (TADMUS) 

Since  1990,  the  TADMUS  program,  which  is  sponsored  by  the  Office  of  Naval  Research,  has  sought 
to  examine  a  variety  of  training  issues  involved  in  group  decision-making  in  the  context  of  a 
shipboard  Combat  Information  Center  (Cannon-Bowers,  Salas,  &  Grossman,  1991).  Johnston, 
Cannon-Bowers,  and  Smith- Jentsch  (1995)  describe  performance  in  such  a  center  as: 

. . .  inherently  complex,  and  often  characterized  by  a  host  of  stressors.  They  include 
rapidly  evolving,  ambiguous  scenarios,  complex,  multi-component  decisions, 
information  overload,  auditory  overload,  command  pressure,  threat,  adverse  physical 
conditions,  and  rapid  interaction  requirements.  Consequently,  training  to  prepare 
operators  for  such  situations  must  be  designed  not  only  to  build  crucial  task  and 
teamwork  skills,  but  also  to  help  teams  be  resilient  to  the  impact  of  stress,  (p.  1) 

This  large  well-organized  program  has  many  thrusts,  but  the  focus  here  will  be  a  project  on  team 
training  and  performance. 

Work  of  their  own  group  and  others  (Klein,  1993)  led  the  TADMUS  group  to  believe  that  there  is 
promise  for  effective  training  interventions  for  Combat  Information  Center  teams.  They  hypothesize 
that: 

•  expert  decision-makers  rely  on  well-organized  knowledge  structures  in  making  decision 

•  over  time,  expert  decision-makers  build  complex  associations  between  situational  cue  patterns 
and  appropriate  strategies/responses 

•  teamwork  skills  are  separate  and  distinct  from  taskwork  skills  and  hence  require  dedicated 
training 

•  expert  teams  have  the  ability  to  adjust  their  strategy  in  accordance  with  task  demands  (stress) 

•  expert  team  members  employ  implicit  coordination  strategies  (i.e.,  without  the  need  to 
communicate)  by  drawing  on  common  or  mutual  knowledge  bases.  (Johnston  et  al.,  1995,  p.  2) 

From  this  it  follows  that  training  needs  to  be  developed  that  will  allow  team  members  to  assess  the 
environment  and  respond  rapidly.  Team  members  must  be  exposed  to  realistic  scenarios  so  they  can 
build  their  knowledge  bases.  For  teams,  this  means  that  team  members  will  require  practice  with 
others  in  the  task  environment  so  that  team-specific  competencies  can  develop  (Cannon-Bowers,  et 
al.,  1995).  Thus,  the  design  of  effective  training  systems  will  rely  heavily  on  establishing 
performance  criteria  related  to  training  objectives  (Hall,  Dwyer,  Cannon-Bowers,  Salas,  &  Volpe, 
1993).  Measures  must  be  accurate  and  sensitive,  and  must  generate  corrective  feedback  as  well 
(Cannon-Bowers  &  Salas,  in  press). 


22 


4.0  CONCLUSIONS  AND  RECOMMENDATIONS 


4.1  SUMMARY  OF  FINDINGS 

This  Review  &  Analysis  begins  by  discussing  the  basic  steps  in  any  decision-making  action.  First 
there  is  awareness  of  the  situation  to  be  addressed,  followed  by  information-gathering,  then 
generation  of  a  plan  and  possibly  some  alternative  plans,  a  choice  among  these  alternatives,  and  then 
action.  How  these  steps  differ  when  groups  rather  than  an  individual  are  making  the  decision  is 
discussed,  with  an  emphasis  on  the  benefits  of  group  decision-making,  such  as  increased  cognitive 
resources,  the  ability  to  generate  more  solutions,  and  the  opportunity  for  greater  detection  of  errors. 
When  decision-makers  are  distributed  (geographically  separated),  they  may  have  greater  computer 
and  other  resources  available  to  them  as  a  group,  and  they  may  benefit  from  their  different  visual 
viewpoints.  However,  even  with  new  immersive  technology,  the  coordination  and  communication 
burden  is  increased  when  teams  are  dispersed. 

Given  our  knowledge  about  how  distributed  collaborative  decision-making  takes  place,  what 
alternatives  does  the  Air  Force  have  in  terms  of  selecting  and  training  tomorrow’s  leaders?  The  state 
of  the  science  regarding  selection  based  on  personality  characteristics  and  aptitudes  for  decision¬ 
making  is  not  optimistic.  Although  there  has  been  a  great  deal  of  research  on  personality 
characteristics  and  group  performance,  the  findings  are  for  the  most  part  too  situation-specific  to 
afford  much  utility  to  the  Air  Force.  However,  there  is  a  plethora  of  literature  that  suggests  one  very 
important  characteristic  about  the  effective  decision-maker,  and  that  is,  the  effective  decision-maker 
is  an  experienced  decision-maker.  This  emphasis  implies  to  the  Air  Force  that  training  in  decision¬ 
making  is  essential.  Expertise  can  be  obtained  through  real-life  experience  in  decision-making,  but 
occurs  much  more  safely  and  efficiently  through  structured  training  exercises.  A  variety  of 
researchers  in  a  variety  of  contexts  are  investigating  many  of  the  important  parameters  in  group 
decision-making  training.  The  following  paragraphs  summarize  the  major  military  training  research 
programs  that  focus  on  group  decision-making. 

4.1.1.  Army  Research 

Important  programs  of  interest  to  the  Air  Force  include  an  Army  program.  Modeling  Army  C  at  the 
Brigade  Level,  which  focuses  on  the  allocation  of  personnel  resources  in  a  C2  Center.  A  baseline 
work  flow  based  on  a  Maneuver  Brigade  Tactical  Operations  Center  was  the  basis  for  this  model.  A 
tiered  approach  was  used  to  form  an  analysis  framework.  In  the  first  tier,  the  workflow  and  mental 
processing  demands  were  established.  Each  subsequent  tier  examined  variations  for  workload 
comparisons.  The  second  tier  posed  the  question,  “What  effect  would  software  have  on  staff 
processes,  the  cognitive  skills  required,  the  type  of  decisions  imposed  by  the  software,  number  of 
people  required,  and  how  they  interact?”  From  these  data,  the  third  tier  adds  the  effect  of 
environmental  stress  factors,  such  as  heat,  noise,  and  vibration.  For  instance,  the  effect  of  noise  on 
group  interaction  might  be  examined.  Tier  four  examines  the  effects  of  removing  face-to-face 
interaction  on  group  processes.  The  fifth  tier  examines  imperfect  communications  such  as  the  effect 
of  data  exchange  rates  and  limited  voice  communication.  Using  this  tiered  approach  allows  the 
researchers  to  isolate  the  effects  of  specific  variables.  Data  are  being  gathered  at  this  time  to  assess 
the  effects  of  the  different  variables  on  C2  group  performance. 
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4.1.2  Klein  and  Associates  Research 


Much  of  the  work  on  designing  training  programs  for  effective  group  decision-making  has  been 
accomplished  by  Gary  Klein,  in  conjunction  with  several  associates.  These  researchers  based  their 
training  program  principles  on  their  observations  of  expert  and  inexpert  decision-making  groups.  The 
Klein  group  emphasizes  the  importance  of  these  variables  as  characteristics  of  expert  decision¬ 
making  units: 

Team  Identity  is  the  extent  to  which  a  team  sees  itself  as  a  unit  that  is  interdependent,  and  uses  that 
perspective  during  team  decision-making.  Team  Conceptual  Level  represents  the  team  as  an 
intelligent  entity.  It  includes  the  group’s  ability  to  think,  solve  problems,  and  made  decisions, 
collectively.  It  also  refers  to  the  information  explicitly  known  by  all  team  members.  Team  Self- 
Monitoring  is  really  metacognition,  and  includes  monitoring  and  regulating  team  performance,  on 
such  variables  as  ability  to  allocate,  monitor,  and  re-prioritize. 

4.1.3  Air  Force  Research 

One  of  the  Air  Force’s  collaborative  distributed  processes  projects  seems  to  have  taken  a  step  back,  to 
a  more  theoretical  examination  of  the  factors  involved  in  group  work.  The  work  of  McNeese  and 
others  focuses  on  a  more  academic  approach  which  seeks  to  investigate  and  define  the  mechanics  of 
how  people  in  groups  work  together.  This  way  of  dealing  with  decision-making  training  may  prove 
to  be  a  good  choice,  since  its  integrated  approach  to  research  and  operations  concerns  may  generate 
some  solid  training  principles.  Future  work  in  the  Living  Lab  will  focus  on  natural  modes  of 
interaction  among  remote  and  collated  sites. 

A  more  applied  Armstrong  Laboratory  program  is  called  C3  STARS — C  Simulation,  Training  and 
Research  systems.  Elliott,  Neville,  and  Dalrymple  (1996)  focus  on  factors  relating  to  Airborne 
Warning  and  Control  System  (AWACS)  Weapons  Director  team  performance  as  Weapons  Directors 
participate  in  an  air  campaign  scenario  which  includes  time  pressure,  high  complexity,  and  ambiguity. 
Imbedded  ambiguous  events  allow  the  monitoring  of  team  communications  and  decision-making. 
Communication,  coordination,  and  situational  awareness  are  assessed  relative  to  performance  and 
mission  accomplishment.  This  effort  seeks  to  determine  how  a  good  team  maintains  communication 
and  situational  awareness  under  such  circumstances. 

4.1.4  Navy  Research 

Since  1990,  the  TADMUS  program,  which  is  sponsored  by  the  Office  of  Naval  Research,  has  sought 
to  examine  a  variety  of  training  issues  involved  in  group  decision-making  in  the  context  of  a 
shipboard  Combat  Information  Center.  Performance  in  such  a  center  is  complex  and  stressful. 
Situations  are  characterized  as  rapidly  evolving,  ambiguous  scenarios,  complex,  multi-component 
decisions,  information  overload,  auditory  overload,  command  pressure,  threat,  adverse  physical 
conditions,  and  rapid  interaction  requirements.  The  focus  of  this  Navy  program  is  to  devise  training 
programs  that  not  only  build  crucial  task  and  teamwork  skills,  but  also  help  teams  be  resilient  to  the 
impact  of  stressors. 

This  Review  and  Analysis  sought  to  examine  the  science  of  decision-making,  and  to  determine  how 
the  Air  Force  could  exploit  this  discipline  in  the  future  battlespace  and  other  military  operations.  The 
most  important  messages  of  this  Review  &  Analysis  for  the  Air  Force  are: 
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•  Research  on  the  differences  between  group  and  individual  decision-making  indicates  that  the 
biggest  benefit  of  collaborative  group  decision-making  is  the  group’s  increased  cognitive 
resources,  compared  to  that  of  a  single  individual. 

•  The  greatest  detriment  to  collaborative  distributed  decision-making  is  that  we  must  rely  on 
technology  rather  than  face-to-face  interactions,  and  subtleties  of  human  communication  may  be 
lost. 

•  The  best  predictor  of  good  decision-making  is  the  experience  of  the  decision-maker,  the 
implication  being  that  training  for  decision-making  is  paramount. 

•  While  personality  and  other  selection  indices  of  the  future  may  have  better  success,  current  tools 
do  not  offer  a  great  deal  of  information  about  who  will  be  a  good  decision-maker  in  general. 

•  All  services  are  engaged  in  research  into  what  variables  are  important  in  a  group  decision-making 
training  program. 

4.2  IMPLICATIONS  FOR  FUTURE  AIR  FORCE  JOB  SKILLS 

The  future  Air  Force  will  need  to  incorporate  many  aspects  of  group  communications  and 
collaboration  into  performance  of  its  tasks.  Excellent  decision-making  skills  will  have  to  be 
developed  in  training  programs,  including  efforts  to  increase  situation  awareness.  Although  not  the 
focus  of  this  Review  and  Analysis,  many  other  skills  will  be  concomitant  to  group  decision-making 
performance.  For  instance,  Air  Force  personnel  must  be  able  to  communicate  across  Services, 
nationalities,  and  cultures.  To  accomplish  collaborative  distributed  work,  Air  Force  personnel  must 
also  be  able  to  use  electronic  media  for  information  transmission.  Future  Air  Force  personnel  will 
have  to  know  how  to  operate  and  maintain  complex  computer  communication  systems,  and  how  to 
find  the  essential  information  for  a  given  situation.  Stress  management  will  be  essential  so  that  teams 
can  function  efficiently  under  the  duress  of  emergency  situations. 

4.2.1  Recommended  Group  Decision-making  Training  Program 

An  ideal  training  program  for  the  Air  Force  to  implement  would  incorporate  three  aspects: 
communication,  situation  awareness,  and  decision-making. 

4.2.1. 1  Communication  Skills  Training 

A  specific  set  of  skills  that  have  a  big  impact  on  group  processes  can  be  taught  as  communication 
skills.  Research  presented  in  this  Review  &  Analysis  indicated  that  as  groups  gain  experience,  they 
learn  efficient  methods  of  communication.  Teaching  group  communication  skills  should  speed  up  the 
training  of  general  group  decision-making  programs.  The  focus  of  communication  skills  training 
should  be  on: 

•  Narrow  conversational  focus 

•  Clear  communication  of  definition,  plans,  strategies 

•  Stylized  speech,  predictable  wording 

Communication  skills  can  be  using  such  templates  as  clarity  and  narrow  focus.  Air  traffic  and  pilot 
stylized  speech  can  keep  meanings  clear  and  concise.  Better  communication  allows  teams  to  share 
mental  models  and  to  keep  one  another  up-to-date  on  plans  and  plan  revisions. 

International  groups  pose  a  whole  entirely  new  set  of  issues.  Little  research  was  found  on  specific 
problems  that  might  need  to  be  addressed  as  the  future  battlefield  includes  cooperative  military- 
information  efforts  among  nations,  but  there  are  some  extant  military  programs  that  address  cross- 
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cultural  issues  in  general.  The  emerging  importance  of  the  Far  East,  East  Europe,  and  third  world 
countries  makes  diversity  and  cross-cultural  issues  of  prime  importance.  The  Navy  has  Personal 
Response  Programs  and  the  Air  Force  has  the  USAF  Special  Operations  School  at  Hurlbert  Field,  FL, 
which  offers  special-operations-related  education,  including  cross-cultural  communications.  DTIC 
documents  indicate  DoD  diversity  training  programs  for  Iran,  Thailand,  Central  America,  Greece,  and 
Arab  countries.  In  addition  to  this  international  focus,  the  corporate  cultures  of  the  different  militaiy 
services  differ,  so  research  needs  to  address  better  ways  for  military  personnel  to  communicate  across 
branches. 

4.2.1.2  Situation  Awareness  Training 

One  area  that  needs  further  investigation  and  development  is  that  of  situational  awareness  training. 
Salas  et  al.  (1995)  say  there  is  a  dearth  of  training  strategies  for  increasing  group  situation  awareness, 
due  in  part  to  a  lack  of  definitional  clarity.  As  the  science  progresses,  an  ideal  Air  Force  training 
program  would  include  such  a  segment,  addressing,  as  suggested  by  Klein  et  al.  (1993),  these  topics: 

•  Recognizing  cues 

•  Strengthening  diagnostic  and  risk  assessment  skills 

•  Increasing  metacognition 

•  Encouraging  strategy  development 

•  Understanding  which  conditions  are  a  best  match  for  particular  strategies 

4.2.1.3  Decision-Making  Skills 

As  the  research  into  the  training  of  decision-making  advances,  the  Air  Force  will  be  able  to 
incorporate  the  best  suggestions  into  its  training  program.  The  emphasis  of  Orasanu  (cited  in 
Kokorian,  1995)  might  serve  as  a  starting  point,  urging  that  good  decision-making  teams,  in  addition 
to  situation  awareness,  focus  on  planfulness,  shared  mental  models,  and  resource  management. 

4.3  RECOMMENDED  RESEARCH 

Comparing  the  vision  of  the  future  Air  Force  with  existing  selection  and  training  programs  allows  us 
to  discern  the  presence  of  some  areas  that  will  need  attention.  The  DoD  programs  and  research 
efforts  are  making  great  progress  in  focusing  attention  on  distributed  collaborative  decision-making, 
and  in  the  creation  of  measures  of  group  decision-making  effectiveness.  These  research  programs 
need  to  continue,  so  that  training  is  administered  to  every  person  who  might  be  involved  in  group 
decision-making  in  the  context  of  his  or  her  job. 

Research  into  group  decision-making  responses  has  made  such  great  strides  over  the  last  decade.  To 
be  more  applicable  to  military  training  and  selection,  it  is  recommended  that  research  continue, 
especially  in  the  areas  of: 

•  basic  abilities  and  personality  characteristics  and  how  they  influence  decision-making 

•  tailored  personality  index  development 

•  cost-effective  selection  or  training  of  situational  awareness  skills 

•  documenting  effective  decision-making  strategies  under  stress 

•  group  composition  and  group  process  effects 
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4.4  CONCLUSIONS 


The  opening  paragraph  of  this  section  summarized  the  implications  for  future  Air  Force  personnel 
activities.  Again,  the  major  skills  necessary  from  the  point  of  view  of  this  Review  &  Analysis  are 

•  group  decision-making  skills,  including  situation  awareness 

•  communication  skills 

•  cross-cultural  and  diversity  skills 

•  computer  expertise 

Manpower,  Personnel,  and  Training  (MPT)  research  needs  to  determine  how  best  these  skills  can  be 
generated.  The  two  choices  are  of  course,  personnel  selection  and  personnel  training.  The  research 
points  to  the  efficacy  of  training  rather  than  selection  of  personnel,  but  both  can  be  implemented  by 
the  Air  Force.  The  first  questions  for  MPT  research  are  the  following: 

•  If  tools  were  available  to  better  screen  good  decision-makers,  would  selection  be  more  cost 
effective  than  training? 

•  What  are  the  most  efficient  ways  to  train  these  skills? 

•  What  is  a  reasonable  interval  for  refresher  training? 

•  How  can  these  skills  be  best  maintained? 
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APPENDIX  A,  BRIEFING 


Distributed  Collaborative  Communications  &  Decision-Making  Briefing 

Presented  by  Joyce  Cameron 
30  Jun  97 


Developed  by 

Joyce  A.  Cameron  and  Frank  C.  Gentner 

Based  on  the  CSERIAC  Review  and  Analysis,  Technological  Impact  on  Future  Air 
Force  Personnel  &  Training:  Distributed  Collaborative  Decision-Making ,  authored  by 

Barbara  Palmer 
Frank  C.  Gentner 
Joyce  A.  Cameron 
Jennifer  I.  Soest 
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*  Based  on  CSERIAC  Review  and  Analysis:  Technological  Impact  on  Future  Air  Force  Personnel  &  Training:  Distributed  Collaborative  Decision-Making  by 
Barbara  Palmer,  Frank  Gentner,  Joyce  Cameron,  and  Jennifer  Soest. 

Note:  this  forecast  of  the  impacts  of  distributed  communications  and  decision-making  is  derived  from  scientific  literature,  DoD,  and  Air  Force 
planning  documents.  The  statements  herein  contain  an  element  of  speculation  and  do  not  necessarily  imply  future  official  plans  or  policy. 
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About  CSERIAC 


The  Crew  System  Ergonomics  Information  Analysis  Center  (CSERIAC)  is  the  gateway 
to  worldwide  sources  of  up-to-date  human  factors  is  the  gateway  to  worldwide  sources 
of  up-to-date  human  factors  and  ergonomics  information  and  technologies  for 
designers,  engineers,  researchers,  and  human  factors  specialists.  CSERIAC  provides 
a  variety  of  products  and  services  to  government,  industry,  and  academia  promoting 
the  use  of  human  factors  and  ergonomics  in  the  design  of  human-operated  equipment 
and  systems. 

CSERIAC’s  primary  objective  is  to  acquire,  analyze,  and  disseminate  timely  information 
on  human  factors  and  ergonomics.  On  a  cost-recovery  basis,  CSERIAC  will: 

•  Distribute  human  factors  and  ergonomics  technologies  and  publications 

•  Perform  customized  bibliographic  searches  and  reviews 

•  Prepare  state-of-the-art  report  and  critical  reviews 

•  Conduct  specialized  analyses  and  evaluations 

•  Organize  and  conduct  workshops  and  conferences 

CSERIAC  is  a  United  States  Department  of  Defense  (DoD)  Information  Analysis  Center 
administered  by  the  Defense  Technical  Information  Center,  Fort  Belvoir,  VA,  technically 
managed  by  the  Armstrong  Laboratory  Human  Engineering  Division,  Wright-Patterson 
Air  Force  Base,  OH,  and  operated  by  the  University  of  Dayton,  Dayton, OH. 
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